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METHOD AND SYSTEM FOR AUTOMATICALLY DISPLAYING AN 
IMAGE AND A PRODUCT IN A PAGE BASED ON CONTEXTUAL 
INTERACTION AND METADATA 

Field of the Invention 

5 This utihty application relates generally to distributing images over a network, 

and more specifically, to employing contextual interaction information in combination 
with information associated with a selected image that is distributed over a network for 
enabling a representation of an available product to be displayed with the selected image 
to a user. 

p 10 Background of the Invention 

S Previous methods have employed several factors to determine the products and 

prices for images ordered over a network. These factors include the content or subject of 
M the image itself, the resolution size, the speed of delivery, the customer's particular use 
and contract conditions imposed by the actual owner of the image. Often, different 
1 5 methods for determining the value of products for a selected image have tended to fix the 
yj price of one or more of these factors for a particular type of customer such as personal, 
r|j business and educational. However, none of the prior art methods have provided for 
S associating information with an image that indicated which products were available for 
that particular image. Typically, different types of products were separately displayed 
20 and only after a user chose a particular type of product, could the user determine if the 
chosen product was available for a selected image. 

Also, in the past, point of entry pages at web sites were not automatically 
customized to present products and options related to the particular interests of repeat 
users. Instead, generic point of entry pages were used and extensive navigation was 
25 required by repeat users to locate other pages with information and products of particular 
interest to the repeat users. 
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The invention provides for storing images along with information indicating their 
available products in a database. When one of the stored images is selected, the 
information associated with the selected image is used in combination with contextual 
interaction information to display each available product. Alternatively, when a 
5 particular product is selected, the invention provides for employing the contextual 
interaction information to display at least a portion of the images that are available for 
use with the selected product. Additionally, the invention provides for storing user 
behavior information for each session and creating a profile for each repeat user based on 
this information. When a repeat user returns to a web site, the invention employs the 
10 profile to automatically customize the point of entry page and offer products and options 
% that match the particular interests of the repeat user. 

^ Summary of the Invention 

U: In accordance with the invention, a method is provided for enabling a product 

% associated with an image to be obtained by a user. In response to receiving contextual 
:i 15 interaction information, an image is displayed. The image is associated with information 
yj indicating each product that is available for the image. This information is employed to 
^i] automatically generate a representation of each product that is available for the image. 
!rf The image and the representation of each product that is available for the image is 
automatically displayed in a page. When the representation of an available product is 
20 selected, the available product for the image is provided to the user. Additionally, the 
image can be stored in a database that may be relational or object-oriented. The image 
can be formatted and stored in the database as a picture in any one of several formats 
including JPEG, TIFF, GIF and PIC. Alternatively, the image may be formatted and 
stored in the database as a movie in any one of several formats including MPEG, QTM 
25 and AVI. 

In accordance with other additional aspects of the invention, information 
associated with the image can be stored in a file that includes the image data. A facility 
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can be employed to create an IPTC format for information associated with the image that 
is included in the same file as the image data. 

In accordance with still other additional aspects of the invention, information 
associated with the image may be stored in a file that is separate from the file that 
5 includes image data. A facility may be employed to associate an SQL attribute with the 
image. The SQL attribute corresponds to a script that enables functionality of at least the 
display of the image and a representation of each available product. 

In accordance with yet other additional aspects of the invention, the information 
associated with the image includes image related data such as tag and business. Tag data 
10 can include title, location, date, original author, model release, subject matter 
y information and name of a subject in the image. Also, business data may include source, 
jfl contract, royalty, territory, contract expiration and batch. Additionally, the information 
associated with the image can identify a type of available product. 

In accordance with other additional aspects of the invention, different types of 
"f45 representation for each available product may be displayed, including picture, video, 
Q gr^hical image, graphical icon and text. Also, the page can be an HTML page and the 
S image and each representation of an available product may be displayed in a graphical 
user interface. Additionally, the display of the image and each representation of an 
□ available product can be based on a combination of user information, product 
20 information, image information and contextual interaction information. 

In accordance with still other additional aspects of the invention, the image stored 
in the database is a cut-down version of the original image that has a size less than the 
full size of the original version of the image. 

In accordance with other additional aspects of the invention, a non-visible 
25 watermark is included in the image that can be identified with an application program. 
Also, a visible watermark can be included with the image. 

In accordance with yet other additional aspects of the invention, other types of 
data may be associated with image data stored in the database, including video, sound 
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and graphic. The sound data can have any one of several formats, including WAV and 
MP3. 

In accordance with other additional aspects of the invention, an agent program 
can be used to automatically determine the image and available products to be displayed. 
5 In accordance with still other additional aspects of the invention, a method is 

provided for using contextual interaction information to enable a product associated with 
an image to be ordered over a network. A search can be entered in a point of entry page, 
which is employed to search a database that includes a plurality of images. The result of 
the search is displayed in a result page. An affirmative result causes at least one image 
10 related to the search to be displayed in the result page. In response to a selection of a 
displayed image in the result page, a product page that includes the representation of 
each available product for the selected image is displayed. The different types of 
available products include picture, electronic postcard, poster, screen saver, wallpaper, 
calendar, stationary, invitation, presentation, slideshow, puzzle and cup. In response to a 
•4^^ 15 selection of the representation of the available product in the product page, a create page 
that includes a representation for each option associated with the selected product is 
displayed. The options for the selected product include at least one template, and the 
J typ^ of use such as single, free and subscription. In response to a selection of the 
□ representation of an option associated with the selected product, a transaction page that 
20 includes each option for a transaction to order the selected product is displayed. Also, in 
response to the selection of the subscription type of use, a subscription page that includes 
each option for selecting a subscription membership is displayed such as individual, 
business and institutional. Additionally, options for the transaction to obtain the selected 
product include send, print, purchase and add to virtual shopping cart. 
25 In accordance with additional aspects of the invention, the search of the database 

may be subject based and in another embodiment, the search can be product based. For a 
product based search, options associated with an available product is displayed in 
another page. 
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In accordance with other additional aspects of the invention, behavior 
information and preferences in each session for each user can be generated and stored in 
a profile corresponding to each user. The profile can be employed to customize the 
search result fi-om the database for a corresponding current user. The profile may be 
used to customize the display of the available product for a corresponding current user. 

hi accordance with yet other additional aspects of the invention, a temporary 
identifier is assigned to a current user. The temporary identifier is employed to 
determine when the current user is a repeat user which has a previously assigned 
permanent identifier. When tiie determination is affirmative, the behavior information 
for the current user is added to a previously created profile for the repeat user and the 
temporary identifier for the current user is changed to the permanent identifier assigned 
to the repeat user. However, when the determination is negative, the behavior 
information for the current user is added to another profile created for the current user 
and the temporary identifier is changed to another permanent identifier tiiat is assigned to 
the current user. The temporary and permanent identifiers may be included with a 
Cookie in an HTTP Response. 

In accordance with other additional aspects of tiie invention, a method is 
provided for enabling a product associated with a sound to be provided to a user. The 
sound is formatted and stored in a database. The sound format includes WAV and MPS. 
Other types of data may be associated with the sound stored in the database, including 
video, picture and graphic. 

In accordance with other additional aspects of the invention, a system which 
implements substantially the same fimctionality in substantially the same manner as the 
methods described above is provided. 

In accordance with yet other additional aspects of this invention, a computer- 
readable medium that mcludes computer-executable mstructions may be used to perform 
substantially the same methods as those described above is provided. 
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Brief Description of the DrawinRs 
The foregoing aspects and many of the attendant advantages of this invention 
will become more readily appreciated of Union or if at away buses run as the same 
become better understood by reference to the following detailed description, when taken 
5 in conjunction with the accompanying drawings, wherein: 

FIGURE 1 shows a logical overview of the actions employed to create a 
relational database of images and associated data; 

FIGURE 2 illustrates a logical overview for associating an SQL attribute with a 
cut-down image in a relational database; 
10 FIGURE 3 shows a logical overview for associating IPTC data with a cut-down 

image in a relational database; 

FIGURE 4A illustrates a flat database file for a cut-down image associated with 
an SQL attribute; 

FIGURE 4B shows a flat database file for a cut-down image associated with 
15 IPTC data; 

FIGURE 5 illustrates a logic overview for a subject based search of a relational 
database; 

FIGURE 6 shows a logic overview for a product based search of a relational 
database; 

20 FIGURE 7 A illustrates an exemplary point of entry page for a subject based 

search of a relational database; 

FIGURE 7B shows an exemplary point of entry page for a product based search 
of a relational database; 

FIGURE 8 illustrates an exemplary results page for a search of a relational 
25 database; 

FIGURE 9 shows an exemplary transaction page for a search of a relational 
database; 
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FIGURE 10 illustrates an exemplary product page for choosing a product 
associated with an image returned from a subject based search of a relational database; 

FIGURE 1 1 shows an exemplary product page for choosing a product action 
associated with a product for an image returned from a product based search of a 
relational database; 

FIGURE 12 illustrates an exemplary create page for choosing the output for a 
selected product; 

FIGURE 13 shows an exemplary subscription page for enabhng a user to select a 
subscription to a relational database of images; 

FIGURE 14A illustrates an exemplary code fragment for an HTTP Request with 
a Cookie that includes a permanent chent identifier; 

FIGURE 14B shows an exemplary code fragment for an HTTP request without a 
Cookie; 

FIGURE 14C illustrates an exemplary code fragment for an HTTP Response 
with a Cookie that includes a temporary client identifier; 

FIGURE 14D shows an exemplary code fragment for an HTTP Response with a 
Cookie that includes a permanent client identifier; 

FIGURE 15 illusfrates a system overview for data centers with servers that are 
remotely located from a client; 

FIGURE 16 shows an exemplary computer system for a chent; 

FIGURE 17 illusfrates an architectural overview of the objects referenced by an 
artificial intelligence engine; and 

FIGURE 18 shows a flow chart representing the logic employed by the artificial 
mtelligence engine to generate the display of the image and related product. 

Detailed Description of the Preferred Embodiment 
The invention provides a method, system and computer-readable medium for 
displaying an image and/or a product based on the context of user interaction. Images 
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with related information are stored in a relational database and using the information to 
order available products for a selected image over a network. Tag information may be 
associated with an image in the database, which can include title, location, date, original 
author/photographer and the name of the subject. A unique image identifier for each 
image is stored in the relational database. Also, business information can be associated 
with the image that may include identification of the source of the original image, batch 
number and contract limitations on the use of the image. Additionally, the invention 
may be used to browse the relational database with product or image based searches, 
subscription "pushes" and hyperlinks that are associated with images, products or 
categories. 

In one embodiment, the invention employs a Hyper Text Transport Protocol 
(HTTP) Response with a Cookie to store a copy of a chent identifier at the client during 
a network session between the invention and the client. When a cHent accesses the 
invention, information associated with the chent's behavior and preferences can be 
stored for each session in a profile that may include the topic of each search and each 
purchased product. A repeat chent's profile can be employed to customize a graphical 
user interface such as a point of entry page with particular options and products that are 
related to the client's behavior. 
Logic Overview 

In FIGURE 1, an overview 100 shows the logical actions for creating and storing 
digital data representing a copy of an original image. Moving fi-om a start block, the 
logic flows to a block 102 where a digitizer is employed to generate raw binary (digital) 
data firom a selected original image. It is envisioned that different types of digitizers, 
e.g., a digital camera and a scanner, could be employed to create binary image data from 
original images. The logic advances to a block 104 where a unique image identification 
number (image ID) is assigned to the raw binary image data. The image ID is employed 
to identify a copy of the binary image data, which is stored in a data warehouse. The 
logic steps to a block 106 where the image ID number is employed by a facility to create 
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a particular entry in a relational database. In one embodiment, a third party facility such 
as MICROSOFT CORPORATION'S SEQUEL SERVER VERSION 7.0 could be 
employed to create the image ID entries and manage the relational database. In yet 
another embodiment, another facility could be used to create and manage image ID 
5 entries in an object-oriented database. 

The logic advances to a block 108 where tag data is associated with the image ID 
entry in the database. The tag data provides information related to the originally selected 
image. For example, the tag data may include the location, date, original 
author/photographer, name(s) of the subject(s), model release, subject matter information 

10 and title of the original image. The logic moves to a block 110 where business data is 
associated with the image ID entry in the database. The business data may include 
information identifying the source, contract, royalty, territory, contract expiration and 
batch number associated with the original image that is used to create the binary image 
data. Batch numbers are identifiers that are typically assigned to a group of original 

1 5 images that were purchased or licensed from a third party. 

The logic flows to a block 112 where at least one cut-down digital image is 
generated from the binary image data. In one embodiment, several different sizes of cut- 
down images are created with resolutions less than the resolution of the image 
represented by the binary image data. For example, cut-down images that have 250 by 

20 250, 640 by 480 and 1280 by 1024 lines of resolution could be created from binary 
image data representing an image with 10,000 by 10,000 lines of resolution. The logic 
steps to a block 114 where the cut-down image(s) are associated with the image ID entry 
in the database. Next, the logic flows to an exit block and terminates. 

In FIGURE 2, an overview 118 shows the logical actions for associating server 

25 search language (SQL) attributes with a cut-down image in a Joint Photographic Experts 
Group (JPEG) format that is associated with an image ID entry in a database. The logic 
moves from a start block to a block 118 where a faciUty is employed to provide an 
invisible digital watermark for each cut-down image. In one embodiment, a third party 
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facility such as provided by the DIGMARC CORPORATION may be used to create the 
invisible digital watermark for a cut-down image. The logic advances to a block 120 
where a facility compresses each cut-down image that includes an invisible digital 
watermark and creates a corresponding JPEG image. Optionally, the invention may 
5 create a corresponding image in any one of a pluraUty of different image formats, 
including graphics interchange format (GIF), tag image file format (TIFF) and pictor 
(PIC). 

The logic flows to a block 122 where a facility creates visible digital watermarks 
that are added to each JPEG image. The logic steps to a block 124 where the JPEG 

10 image is associated with a corresponding image ID entry in the database. The logic 
moves to a block 126 where SQL attributes are created and associated with the image ID 
entry. The SQL attributes correspond to scripts that enable particular functions for a 
JPEG image associated with a particular image ID entry. For example, these scripts 
could enable the functionality for sending, printing, and viewing a JPEG image 

15 associated with SQL attributes. Also, the scripts could enable the functionality for 
generating an electronic postcard, purchasing a poster, limiting the syndicated use of a 
JPEG image and controlling the location for displaying the JPEG image. In one 
embodiment, the SQL attributes are automatically generated by a facility based on the 
tag data and business data associated with the image ID entry. In another embodiment, 

20 the SQL attributes are manually created and associated with the image ID entry. Next, 
the logic moves to an exit block and terminates. 

In FIGURE 3, an overview 128 shows the logical actions for employing a facility 
to add information in an International Press and Telecommunications Council (EPTC) 
format to a JPEG image associated with an image ED entry in a database. The IPTC 

25 format provides a hierarchical structure for information that is added to a file containing 
the JPEG image. 

The logic moves from a start block to a block 130 where a facility is employed to 
provide an invisible digital watermark for each cut-down image. The logic advances to a 
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block 132 where a facility compresses each cut-down image with an invisible digital 
watermark and creates a corresponding JPEG image. 

The logic flows to a block 134 where a faciUty creates a visible digital watermark 
that is added to each JPEG image. The logic steps to a block 136 where a facility is 
5 employed to generate information in an IPTC format that is added to each file containing 
each JPEG image. The IPTC formatted information may include the image ID, tag data 
and business data. Also, this information may be used to define the functionahty of each 
JPEG image. The logic moves to a block 138 where the JPEG image that includes the 
IPTC formatted information is associated with the image ID entry in the database. Next, 

1 0 the logic jumps to an exit block and terminates. 

In FIGURE 4 A, an overview 140 is shown of a flat file entry in a database having 
fields associated with the image ID, JPEG file and SQL attributes. The presence of a "1" 
in a field associated with an SQL attribute indicates that this attribute's fimctionahty is 
enabled for the image assigned to the image ID and the presence of a "0" indicates that 

15 the fimctionahty is not enabled. In the exemplary embodiment, the fimctionahty for 
send and electronic postcard SQL attributes are enabled and the print SQL attribute is 
not enabled. Additionally, FIGURE 4B shows an overview 142 of a flat file entry in a 
database having fields associated with an Image ID and a JPEG file with IPTC 
information, 

20 Lofflc Overview Of Subject Based Search 

FIGURE 5 shows an overview 146 of a subject based search to order a product 
associated with a selected image in a relational database. Moving from a start block, the 
logic steps to a block 148 where a subject based search is performed on a relational 
database through an interface in a point of entry page. An exemplary interface for 

25 providing a subject based search in a point of entry page 174 is shown in FIGURE 7 A. 

Returning to FIGURE 5, the logic advances from the block 148 to a block 150 
where in response to the subject search, a search engine retrieves images related to the 
searched subject from the relational database and displays relevant images in a search 
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result page. FIGURE 8 shows an exemplary search result page 178 with several 
thumbnail size JPEG images 180 that are relevant to the searched subject. 

In FIGURE 5, the logic flows from the block 150 to a block 152 where in 
response to the selection of an image in the search result page 178, a relatively larger 
5 cut-down version of the selected thumbnail JPEG image and a representation of each 
product available for the selected image are displayed in a product page. FIGURE 10 
illustrates an exemplary product page 196 that displays a relatively larger cut-down 
JPEG image 198 of the selected thumbnail JPEG image 180 and several graphical 
buttons 200 representing various products that are available for the selected image. 

10 Next, the logic steps from the block 152 to a block 154 where in response to the 

selection of a graphical button 200 for a product in the product page 196, the cut-down 
JPEG image 198 and several product options are displayed in another product page. 
FIGURE 1 1 illustrates another exemplary product page 202 that displays the cut-down 
JPEG image 198 of the selected image and several graphical buttons representing 

15 product options, including: (1) a color button 206 for selecting the color of the chosen 
product; (2) a size button 208 for selecting the size of the selected product; (3) a custom 
button 210 for customizing an action for the selected product; and (4) a show all button 
212 for causing the display of all of the product options available for a chosen product. 

In response to the selection of a product option displayed in the other product 

20 page 202, the logic moves to a block 155 where the cut-down JPEG image 198 and other 
product options are displayed in a create page. In FIGURE 12, an exemplary create page 
214 is shown with the other product options represented by graphical buttons. These 
buttons include (1) a template button 218 for selecting any of several product templates 
220; (2) a free button 222 for enabUng a copy of the selected product to be ordered for 

25 free; (3) a single price button 224 for purchasing an individual copy of the selected 
product; and (4) a subscription price/upsell button 226 for a subscribing user to order the 
selected product with the ability to sell (upsell) the selected product to others. 
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The actual cost of the related product for a selected image would be influenced 
by the configuration of the image within the product template. For example, small 
image resolutions might be given away for free with some products such as thumbnail 
image with 150 by 150 lines of resolution. Alternatively, larger image resolutions, e.g., 
5 640 by 480 hnes of resolution and above, would cause a related product such as a screen 
saver to cost $5.00 or more. 

Returning to FIGURE 5, the logic steps from the block 155 to a block 156 where 
in response to the selection of a button in the create page, the cut-down JPEG image 198 
and several transaction options are displayed in a transaction page. FIGURE 9 illustrates 

10 an exemplary transaction page 182 with descriptive text 186 and several transaction 
options that are represented by graphical buttons. These buttons include (1) an add 
button 188 for adding the selected product to a virtual shopping cart for later purchase; 
(2) a purchase button 190 for enabling the purchase of the selected product; (3) a print 
button for printing the selected product such as a calendar or poster; and (4) a send 

15 button for enabling the selected product to be sent to another user. 

In FIGURE 5, the logic moves from the block 156 to a block 158 where in 
response to the selection of a graphical button in the transaction page, the particular 
transaction option associated with the selected button is performed. For example, when 
the purchase button is selected, a transaction is initiated with an electronic commerce 

20 server to facilitate the purchase of the selected product. Next, the logic steps to an exit 
block and terminates. 

Optionally, it is envisioned that a subscriber page could be displayed for enabling 
a user to subscribe to a service that enables various types of uses for selected images and 
products. In FIGURE 13, an exemplary subscriber page 228 illustrates text 230 and 

25 graphical buttons. The text 230 may be used to describe the benefits of subscription. 
The graphical buttons include: (1) an individual subscription button; (2) a business 
subscription button; and (3) an institutional (educational/non-profit organization) 
subscription. Although not shown, when a user's subscription has expired, the invention 
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can display an "expired subscription" button in the subscriber's page. The selection of 
this button would cause a renewal page (not shown), which would enable the user to 
renew and/or modify their expired subscription. 

Additionally, the invention enables updated information related to new products 

5 and images to be "pushed" (sent) to a subscriber. Also, the invention enables the 
presentation of available images and related products to be customized according to the 
type of subscription that a user has purchased/selected. 
Logic Overview of Product Based Search 

FIGURE 6 shows an overview 160 of the logical actions performed in a product 

10 based search for ordering a selected product that is associated with an image in a 
relational database. Moving from a start block, the logic steps to a block 162 where a 
product based search is performed on the relational database through an interface in a 
point of entry page. An exemplary interface for providing a product search in a point of 
entry page 176 is shown in FIGURE 7B. 

15 Returning to FIGURE 6, the logic advances from the block 162 to a block 164 

where in response to the product search, JPEG images that can be provided with the 
searched product are retumed from the relational database and displayed in a search 
resuh page. FIGURE 8 shows an exemplary search result page 178 with several 
thumbnail JPEG images 1 80 that can be provided with the searched product. 

20 In FIGURE 6, the logic flows from the block 164 to a block 166 where in 

response to the selection of an image in the search result page 178, the cut-down image 
and several graphical buttons representing other product options are displayed in the 
other product page. As discussed above, FIGURE 11 illustrates an exemplary 
embodiment of the other product page 202. The logic steps from the block 166 to a 

25 block 168 where in response to the selection of one of the graphical buttons in the other 
product page 202, the cut-down JPEG image 198 and several options for the product are 
displayed in the create page. As discussed above, FIGURE 12 illustrates the exemplary 
create page 214 which includes several graphical buttons that represent other product 
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related options. Looking to FIGURE 6, the logic steps from the block 168 to a block 
170 where in response to the selection of a button in the create page, the cut-down JPEG 
image 198 and several options are displayed in a transaction page. An exemplary 
transaction page 182 is discussed above and shown in FIGURE 9. 
5 In FIGURE 6, the logic moves from the block 170 to a block 172 where in 

response to the selection of a button in the transaction page, the particular option 
associated with the selected button is performed. Next, the logic steps to an exit block 
and terminates. 

Persistent Ghent Identification With HTTP Cookie 

10 HTTP is an application level protocol typically used for transferring data across 

the Internet, e.g., a network data object or server, and it can be specified in a Uniform 
Resource Locator (URL) address that is provided to a browser program. The Hyper Text 
Mark-up Language (HTML) is a simple data format that is used to create hypertext 
documents that are supported by the HTTP protocol. Together, HTTP and HTML have 

1 5 significantly contributed to the creation of the World Wide Web (WWW) on the Internet. 
The WWW is a globally accessible and platform-independent hypermedia information 
system that has become a central access point to applications and services for users 
around the world. 

A Cookie is a general mechanism, i.e., protocol, which server side connections 
20 can use to both store and retrieve information on the client side of the connection. The 
addition of a simple, persistent, client-side state significantly extends the capabihties of 
Internet-based client/server apphcation programs. A server, when returning a HTTP 
object to a chent, may also send a piece of state information which the client may store. 
Included in that state object is a description of the range of Uniform Resource Locators 
25 (URLs) for which the returned state is vaUd. Any future HTTP requests made by the 
cHent which fall in that range will include a transmittal of the current values of the state 
object from the client back to the sender. This state object is called a "Cookie," for no 
compelling reason. 
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The Cookie mechanism provides a powerfiil tool that enables different types of 
application programs to be written for Internet-based environments. For example, a 
service program could use a Cookie to send back registration information and free the 
client from retyping a user identification number for each connection to the service. 
5 Also, an Internet site could store user preferences for a client and have the client supply 
those preferences each time that the client connected to the site. 

Generally, a Cookie is introduced to the client by including information with a 
Set-Cookie coromand in a header as part of an HTTP response. An example of the Set- 
Cookie command included in an HTTP response header is Hsted below. 
10 <HEADER> 

Set-Cookie: NAME = VALUE; expires = DATE; 

path=PATH; domain = DOMAIN _NAME; secure 

</HEADER> 

When a cHent's browser program is requesting resources at a particular URL 
1 5 address from an HTTP server on the Internet, the browser will match the requested URL 
against all of the URLs stored in the cheat's Cookies. If the requested URL matches any 
of the stored URLs, a line containing the name/value pairs of all matching Cookies will 
be included m the HTTP request. An exemplary line in a Cookie for an HTTP request 
could be included as follows: Cookie: NAMEl = OPAQUEJTRINGl; NAME2 = 
20 OPAQUEJTRINGl. 

A Cookie is typically used to save the state of a relationship between a cHent and 
a server with a unique chent identifier. However, in some cases, the saved state of the 
relationship or Cookie may be deleted by the client and/or not persistently stored. For 
example, the chent may employ a Cookie management program that automatically 
25 deletes stored Cookies after each chent-server session or does not allow a server to 
provide a Cookie to the chent. Unfortunately, if the client does not have a previously 
stored Cookie when it makes repeated HTTP requests of a server, the server may not 
initially recognize that a repeat chent is participating in the current session. 
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Since saving a state relationship in a Cookie at a client can be a problem, the 
invention can also monitor the behavior of an imknown or new client to determine if the 
new client is actually a repeat client that did not retain a Cookie previously provided in 
an HTTP response. For example, when the new client enters identifying information in a 
5 point of entry page for a new client, the invention will compare this information to other 
information previously stored in client profiles to determine if the new chent is actually a 
repeat client. Also, when the new chent purchases a product, the invention will compare 
a new chent 's financial transaction information to previously stored information in each 
client profile in order to identify a repeat client. 

10 The invention creates a temporary chent identifier for each new chent that starts a 

session with a server. The temporary user identifier is sent to the chent in a Cookie that 
is included in an HTTP response during the session. However, if the new chent is 
determined to actually be a repeat client during the session, then an HTTP response that 
includes a Cookie with a copy of a previously assigned permanent client identifier for 

15 the repeat client is sent to the repeat client during the session. Alternatively, if the new 
client is not determined to be a repeat client by the end of a session, the invention will 
send an HTTP response with a Cookie to store a copy of a newly assigned permanent 
client identifier which will replace the copy of the temporary client identifier stored at 
the client. 

20 Additionally, as the number of repeat chent sessions increase, the invention 

provides an increasingly customized user interface. The invention employs the 
permanent client identifier to persistently store in an associated client profile the 
historical behavior information associated with a repeat client's behavior for each 
session, e.g., the topic of each search and each purchased product. The invention 

25 employs a chent profile to customize the presentation of a graphical user interface such 
as a point of entry page with particular features and products related to the repeat client's 
preferences. 
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In FIGURE 14A, an exemplary HTTP request 238 with a Cookie that includes a 
permanent chent identifier is shown. FIGURE 14B illustrates an exemplary HTTP 
request 240 without a Cookie identifying the client. FIGURE 14C shows an exemplary 
HTTP response 242 with a Cookie that includes a temporary client identifier. FIGURE 
5 14D illustrates an exemplary HTTP response 244 with a Cookie that includes a 
permanent chent identifier, 

It is envisioned that other methods may be employed to provide a persistent 
identifier for a client. Querystring parameters are another means of persisting a client's 
identity cross many pages. For example, when a dynamic link is selected on a first page, 
10 this page could read in a client's identifier and append the identifier to a query string for 
the URL in the link. A following page would be able to read the URL's appended name- 
value pair (client identifier) for determining the client's identity. 

Additionally, it is envisioned that a client-side application program could be 
downloaded fi*om a server that would work in conjunction with a browsing facility. This 
15 client-side application program could send the client's identifier to a server as necessary 
to persistently identify the chent. 
System Overview 

In FIGURE 15, an overview 300 illustrates an exemplary system for acquiring 
and storing a digitized image and enabhng the functionality of a selected image. At a 

20 data center 316, a source server 302 can be employed to digitize an original image and 
store a copy of the digitized image as binary data. As discussed in greater detail above, 
JPEG images are created and stored in at least one relational database (not shown) which 
can be accessed by an operational server 304. Although not shown at the data center 
316, a facility may be provided for removing artifacts that degrade the quality of an 

25 image generated fi-om the binary image data. It is envisioned that at least a portion of the 
artifact removal process could be performed automatically. The raw binary image data is 
stored in a data warehouse server 306 that may be local or (optionally) remotely located 
from the data center 316. 
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At another data center 308, a personal content server 310, a commercial content 
server 312, an enterprise content server 314 and another relational database (not shown) 
are disposed. The image data stored in the other relational database at the other data 
center 308 mirrors the image data stored in the relational database at the data center 316, 
5 which are coupled to each other by a network. The content servers enable the 
functionality of the invention for a particular type of chent, i.e., personal, commercial 
and enterprise. At predetermined intervals, the content of the other relational database at 
the other data center 308 can be updated with the most recent image ID entries in the 
relational database at the data center 316. It is understood that the disposition of 
10 relational databases with mirrored entries at two separate data centers provides 
P significant fault tolerance and image data redundancy. A client 10, the data centers 308 
yi and 316 and an electronic commerce server 320 are coupled to each other over a wide 
1 J area network 318 such as the Internet. 

f"^ FIGURE 16 illustrates a system for the client 10 comprising components of a 

SI 5 compxxter suitable for executing an application program embodying the present 
□ invention. In FIGURE 16, a processor 12 is coupled bi-directionally to a memory 14 
^ that encompasses read only memory (ROM) and random access memory (RAM). ROM 
[y is typically used for storing processor specific machine code necessary to bootup the 
p computer comprising client 10, to enable input and output functions, and to carry out 
20 other basic aspects of its operation. Prior to running any application program, the 
machine language code comprising the program is loaded into RAM within memory 14 
and then executed by processor 12. Processor 12 is coupled to a display 16 on which the 
visuahzation of the HTML response discussed above is presented to a user. Often, 
programs and data are retained in a nonvolatile memory media that may be accessed by a 
25 compact disk-read only memory (CD-ROM) drive, compact disk-read/write memory 
(CD-R/W) drive, optical drive, digital versatile disc (DVD) drive, hard drive, tape drive 
and floppy disk drive, all generally indicated by reference numeral 18 in FIGURE 16. A 
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network interface 22 couples the processor 12 to a wide area network such as the 
Internet. 

As noted above, the invention can be distributed for use on the computer 
system for the chent 10 as machine instructions stored on a memory media such as a 
5 floppy disk 24 that is read by the floppy disk drive. The program would then typically 
be stored on the hard drive so that when the user elects to execute the application 
program to carry out the present invention, the machine instructions can readily be 
loaded into memory 14. Control of the computer and selection of options and input of 
data are implemented using input devices 20, which typically comprise a keyboard and a 
10 pointing device such as a mouse (neither separately shown). Further details of 
g system for the chent 10 and of the computer comprising it are not ilhistrated, since they 

are generally well known to those of ordinary skill in the art. 
"^J Overview of Artificial Intelligence Engine Logic and Architecture 
y FIGURE 17 illustrates an overview 400 of the object-oriented architecture for 

J'lS implementing the invention with a rules based artificial intelhgence engine 402. The 

□ engine 402 interfaces with an image object 404 that provides access to image metadata, 

ill 

p e.g., image properties, image usage rights and special products available for use with the 
g image. A product object 406 also interfaces with the engine 402. The product object 

^ 406 may provide the engine 402 with access to business preference data such as product 
20 properties and configuration options. The engine 402 interfaces with a user object 408 
that can provide access to user metadata including user rights, behavior (historical), 
preferences and subscription status. Additionally, the engine 402 interfaces with a 
context object 410 that provides contextual interaction data including: (1) the type of 
viewer employed to access a page displaying images and related products such as a 
25 browser or digital canvas; (2) fi-om what place in a web site that a user accessed the 
invention, e.g., a home page, special interest page, subscriber page or third party web site 
page; and (3) from what interaction in a web site that a user accessed the invention, e.g., 
a search, browse, or click through fi*om a hyperlinked object. 
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The engine 402 employs the different types of data provided by the image object 
404, product object 406, user object 408 and context object 410 to perform a rules based 
(artificial intelligence) analysis for determining which images and related products will 
be displayed in a page. In this way, the invention enables a customized display of 
5 images and related products to be provided for each user. 

In FIGURE 18, a flow chart 412 illustrates a logical overview for analyzing 
which images and related products will be displayed in a page to a user. Moving from a 
start block, the logic advances to a block 414 where the contextual interaction of a user 
with the invention is analyzed, such as type of viewing program, hierarchical position of 

10 a user in a web site, search criteria, browsing results and hyperlinked click throughs. 
The logic steps to a block 416 where the logic determines the particular attributes of the 
current user, including rights, behavior, preferences and subscription status of the user. 
Advancing to a block 418, the logic determines the image attributes of the selected 
image, including properties, usage rights and special related products. The logic flows to 

15 a block 420 where the logic determines the attributes of the selected product that is 
related to the selected image. For example, the logic determines the properties and 
options for the selected product. Moving forward, the logic advances to a block 422 
where the logic employs artificial inteUigence (rules based analysis) to generate a display 
of the images and related products. Also, the logic can employ the category of the image 

20 and the actual use of the image to generate a display of the images and products. Next, 
the logic steps to an exit block and the logic flow terminates. 

In another embodiment, an agent program (not shown) may be employed to 
automatically search a relational database to select an image and/or procure a related 
product in accordance with a user request. The agent program may be associated with an 

25 electronic appliance/device that enables the program to select an image and/or product 
from any one of multiple relational databases on a network. The user can employ the 
agent program to automatically choose an image and/or product based on different 
criteria including theme, category, context and subject. The agent program may be 
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configured to automatically search at least one network and gather certain types of 
information in the backgroimd. 

Although subject based and product based searches are discussed above, in 
another embodiment a sound based search and/or a video based search may be 

5 performed in substantially the same way when the relational database of image data also 
includes sound and/or video data. The sound data may be provided in any one of several 
formats, including Motion Picture Expert Group Version 3 (MPEG 3 or MP3) and 
WAV. Also, the video data can be provided in any one of several formats, including 
MPEG, QuickTime (QTM) and Audio Video Interleaved (AVI). In other embodiments, 

10 the invention can store video data or sound data instead of image (picture) data in the 
relational database. In either case, storing the sound/video data and the associated 
information in the relational database, creating relational database sound/video ID 
entries, and subject and product based searching would be performed in substantially the 
same manner as discussed above for a relational database that did include image ED 

15 entries and image (picture) data. 

While the preferred embodiment of the invention has been illustrated and 
described, it will be appreciated that various changes can be made therein without 
departing from the spirit and scope of the invention. 
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The embodiments of the invention in which an exclusive property or 
privilege is claimed are defined as follows: 

1 . A method for enabling a product associated with an image to be provided 
to a user, comprising: 

(a) employing the context of an interaction to determine the image to 
be displayed, the image being displayed and being associated with information 
indicating each product that is available for use with the image; 

(b) automatically employing the information associated with the 
image to generate a representation of each product that is available for use with the 
image; 

(c) automatically displaying the image and the representation of each 
product that is available for use with the image in a page; and 

(d) when the representation of an available product is selected, 
enabUng the available product for use with the image to be provided. 

2. The method of Claim 1, wherein the image is stored in a database. 

3. The method of Claim 2, wherein the image is formatted and stored in the 
database as a picture, the format including JPEG, GIF, TIFF and PIC. 

4. The method of Claim 2, wherein the image is formatted and stored in the 
database as a movie, the format including MPEG, QTM and AVI. 

5. The method of Claim 2, wherein the database is a relational database. 

6. The method of Claim 2, wherein the database is an object oriented 
database. 
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7. The method of Claim 1, wherein the information associated with the 
image is stored in a file that includes image data. 

8. The method of Claim 7, further comprising employing a facility create an 
IPTC format for associated information that is included in the file that includes image 
data. 

9. The method of Claim 2, wherein the information associated with the 
image is stored in the database separate fi-om a file that includes image data. 

10. The method of Claun 9, further comprising associating an SQL attribute 
with the image, the SQL attribute corresponding to a script that enables a fimctionality 
for the image. 

11. The method of Claim 1, wherein the information associated with the 
image includes image related data. 

12. The method of Claim 11, wherein the image related data is tag data that 
includes title, location, date, original author, model release, subject matter information 
and name of a subject in the image. 

13. The method of Claim 1 1, wherein the image related data is business data 
that includes source, contract, batch, royalty, territory and contract expiration. 

14. The method of Claim 1, wherein the information associated with the 
image identifies a type of available product for the image. 

15. The method of Claim 1, wherein the representation of each available 
product includes different types of representation, including picture, video, graphical 
image, graphical icon and text. 

16. The method of Claim 1 , wherein the page is an HTML page. 
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17. The method of Claim 1, further comprising employing a combination of 
user information, image information, product information and contextual interaction 
information to determine the display of the image and the product. 

1 8 . The method of Claim 1 , wherein the image is a cut-down version having a 
size that is less than a full size of an original version of the image. 

19. The method of Claim 1 , further comprising: 

(a) including a non-visible watermark in the image; and 

(b) enabling the non- visible watermark in the image to be identified 
with an application program. 

20. The method of Claim 1, further comprising including a visible watermark 
in the image. 

21. The method of Claim I, further comprising associating another type of 
data with the image data, the other type of data including video, sound and graphic. 

22. The method of Claim 21, wherein the sound has a format, including 
WAV and MPS. 

23. The method of Claim 1, farther comprising an agent program that 
automatically employs contextual interaction information to determine the image to be 
displayed. 

24. The method of Claim 23, wherein the agent program performs a search of 
at least one database on a network to determine the image to be displayed. 

25. A computer readable medium having computer-executable instructions 
for performing the method recited in Claim 1. 
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26- A system for enabling a product associated with an image to be provided 
to a user, comprising: 

(a) a server for implementing logical actions, including: 

(i) in response to receiving contextual interaction information 
from a client, providing the image to the client, wherein the image is associated with 
information indicating each product that is available for use with the image; and 

(ii) in response to a request to provide a product that is 
available for use with the image, enabling the product to be provided; and 

(b) the client for implementing logical actions, including: 

(i) providing the contextual interaction information to the 

server; 

(ii) when the image is received from the server, automatically 
employing the information associated with the image to generate a representation of each 
product that is available for use with the image; 

(iii) automatically displaying the image and the representation 
of each product that is available for use with the image in a page; and 

(iv) when the representation of an available product is selected, 
enabling the available product for use with the image to be provided. 

27. A method for enabling a product associated with an image to be provided 
to a user, comprising: 

(a) employing the context of an interaction to determine a 
representation of the product to be displayed, the representation of the product being 
displayed and being associated with information indicating each image that is available 
for use with the product; 

(b) automatically employing the information associated with the 
product to display at least one image that is available for use with the product; 
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(c) automatically displaying the image and the representation of each 
product that is available for the image in a page; and 

(d) when the representation of an available product is selected, 
enabling the available product for the image to be provided. 

28. The method of Claim 27, wherein the image is stored in a database. 

29. The method of Claim 28, wherein the image is formatted and stored in the 
database as a picture, the format including JPEG, GIF, TIFF and PIC. 

30. The method of Claim 28, wherein the image is formatted and stored in the 
database as a movie, the format including MPEG, QTM and AVI. 

31. The method of Claim 27, further comprising employing a combination of 
user information, product information, image information and contextual uiteraction 
information to determine the display of the image and the product. 

32. The method of Claim 27, wherein the information associated with the 
product includes properties and options. 

33. The method of Claun 27, further comprising associating information with 
the image, including image related data. 

34. The method of Claim 33, wherein the image related data is tag data that 
includes title, location, date, original author, model release, subject matter information 
and name of a subject in the image. 

35. The method of Claim 33, wherein the image related data is business data 
that includes source, contract, royalty, territory, contract expiration and batch. 

36. The method of Claim 33, wherein the information associated with the 
image identifies a type of available product for the image. 
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37. The method of Claim 27, wherein the representation of each available 
product includes different types of representation, including picture, video, graphical 
image, graphical icon and text, 

38. The method of Claim 27, further comprising associating another type of 
data with the product, the other type of data including video, sound and graphic. 

39. The method of Claim 27, further comprising an agent program that 
automatically employs contextual interaction information to determine the image to be 
displayed. 

40. The method of Claim 39, wherein the agent program performs a search of 
at least one database on a network to determine the image to be displayed. 

41. A computer readable medium having computer-executable instructions 
for performing the method recited in Claim 27. 

42. A chent-server based system for implementing the actions in the method 
recited in Claim 27. 

43. A method for enabling a product associated with an image to be ordered 
over a network, comprising: 

(a) displaying the image in response to receiving contextual 
interaction information, image being associated with information indicating each product 
that is available for the image; 

(b) in response to selecting the displayed image, automatically 
employing the information associated with the image to generate a representation of each 
available product for the image; 

(c) automatically displaying the image and the representation of each 

available product for the image in a page; and 
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(d) when the representation of an available product is selected in the 
page, enabling the available product for the image to be ordered by a user. 

44. The method of Claim 43, further comprising entering a value for a search 
in a point of entry page, the value being employed to search a database that includes a 
plurahty of images. 

45. The method of Claim 43, further comprising displaying a result of the 
search in a result page, an affirmative result causing at least one image related to the 
search to be displayed in the resuU page. 

46. The method of Claim 43, further comprising in response to a selection of 
a displayed image, displaying a product page that includes the representation of each 
available product for the selected image. 

47. The method of Claim 46, wherein the available product includes picture, 
electronic postcard, poster, screen saver, wallpaper, calendar, stationary, invitation, 
presentation, slideshow, puzzle and cup. 

48. The method of Claim 43, fiirther comprising in response to a selection of 
the representation of the available product for the selected image, displaying a create 
page that includes a representation for each option associated with the selected product 
for the selected image. 

49. The method of Claim 48, wherein an option for the selected product, 
includes at least one template. 

50. The method of Claim 48, wherein an option for the selected product is a 
type of use for the selected product, the type of use including single, free and 
subscription. 
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5L The method of Claim 48, further comprising in response to a selection of 
the representation of an option associated with the selected product, displaying a 
transaction page that includes each option for a transaction to order the selected product. 

52. The method of Claim 49, further comprising in response to the selection 
of subscription, displaying a subscription page that includes each option for selecting a 
subscription membership including individual, business, institutional and renewal. 

53. The method of Claim 48, wherein an option for the transaction to obtain 
the selected product includes send, print, purchase and add to virtual shopping cart. 

54. The method of Claim 48, wherein each option associated with the 
available product is displayed on another page. 

55. The method of Claim 43, further comprising: 

(a) generating behavior information and preferences in each session 
for each user; and 

(b) storing the behavior information and preferences in a profile, each 
profile being associated with a user. 

56. The method of Claim 55, further comprising employing the profile to 
customize the display of the image, a current user being associated with the profile. 

57. The method of Claim 55, further comprising employing the profile to 
customize the display of the available product, a current user being associated with the 
profile. 

58. The method of Claim 55, further comprising assigning a temporary 
identifier to a current user. 

59. The method of Claim 58, further comprising: 
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(a) employing the temporary identifier to determine when the current user is 
a repeat user, the repeat user having a previously assigned permanent identifier; and 

(b) when the determination is affirmative, adding behavior information for 
the current user to a previously created profile for the repeat user and changing the 
temporary identifier for the current user to the permanent identifier assigned to the repeat 
user; else 

(c) when the determination is negative, adding behavior information for the 
current user to another profile for the current user and changing the temporary identifier 
to another permanent identifier assigned to the current user. 

60. The method of Claim 59, wherein each identifier is included with a 
Cookie in an HTTP Response. 

61. The method of Claim 59, wherein each identifier is included as a 
parameter in a query string associated with a selection of a hyperlink in the page. 

62. The method of Claim 59, wherein each identifier is provided by a 
program associated with the chent. 

63. The method of Claim 43, further comprising associating another type of 
data with the image in the relational database, the other data type including video, sound 
and graphic. 

64. The method of Claim 43, wherein the image is formatted and stored in the 
database as a picture, the format including JPEG, GIF and PIC. 

65 . The method of Claim 43, wherein the image is formatted and stored in the 
database as a movie, the format including MPEG, QTM and AVI. 
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66. A computer readable medium having computer-executable instructions 
for performing the methods recited in Claim 43. 

67. A method for enabling a product associated with an image to be ordered 
over a network, comprising: 

(a) displaying a representation of the product in response to receiving 
contextual interaction information, the product being associated with information 
indicating each image that is available for use with the product; 

(b) in response to selecting the displayed representation of the 
product, automatically employing the information associated with the product to display 
at least a portion of the images available for use with the product; and 

(d) when the image is selected, enabling the product available for use 
with the image to be ordered by a user. 

68. A system for placing an order for a product associated with an image over 
a network, comprising 

(a) a server for implementing logical actions, including: 

(i) in response to receiving contextual interaction information 
from a client, providing the image to the cHent, the image being associated with 
information that indicates each available product for the image; and 

(ii) in response to a request to order the available product for 
the image, enabhng the product to be provided; and 

(b) the chent for implementing logical actions, including: 

(i) providing the contextual interaction information to the 

server; 

(ii) in response to receiving the image, automatically 
employing the information associated with the image to generate a representation of each 
product that is available for the image; 
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(iii) automatically displaying the image and the representation 
of each product that is available for the image in a page; and 

(iv) when the representation of an available product is selected 
in the displayed page, enabling the available product for the image to be ordered from 
the server. 

69. A method for enabling a product associated with a sound to be provided 
to a user, comprising: 

(a) in response to receiving contextual interaction information, 
displaying a representation of the sound, the sound being associated with information 
indicating each product that is available for use with the sound; 

(b) automatically employing the information associated with the 
sound to generate a representation of each product that is available for the sound; 

(c) automatically displaying the representation of each product that is 
available for the sound and the representation of the sound in a page; and 

(d) when the representation of an available product is selected in the 
displayed page, enabling the available product for the sound to be provided to the user. 

70. The method of Claim 69, wherein the sound is formatted and stored in a 
database, the format including WAV and MP3. 

71. The method of Claim 69, further comprising associating another type of 
data with the sound stored in a database, the other type of data including movie, picture 
and graphic. 

72. A computer readable medium having computer-executable instructions 
for performing the method recited in Claim 69, 

73. A client-server system for implementing computer-executable 
instructions for performing the method recited in Claim 69. 
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METHOD AND SYSTEM FOR AUTOMATICALLY DISPLAYING 
AN IMAGE AND A PRODUCT IN A PAGE BASED ON CONTEXTUAL 
INTERACTION AND METADATA 

Abstract of the Disclosure 

The invention provides a method, system and computer-readable medium for 
storing images with related information in a relational database and using the 
information to order available products for a selected image over a network. Tag and 
business information may be associated with an image in the database and a unique 
image identifier is generated for each image. Different types of searches of the relational 
database may be performed including product based and image based. Affirmative 
results fi-om searches are returned as images and/or products that are displayed in pages. 
The retumed images also include information indicating available products. The 
selection of a retumed image causes a cut-down version of the image and all of the 
available products to be displayed in the same page. When an available product is 
selected, a server enables a transaction for a user to order the selected product. A Hyper 
Text Transport Protocol (HTTP) Cookie is employed to store a copy of a client identifier 
at a client. Information associated with the client's behavior for each session is stored in 
a profile that is used to customize the particular options and products that are displayed 
to the user. A combination of user information, product information, image information 
and contextual interaction information is used to determine the display of the images and 
the products. 
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